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Abstract 

We study the class of self-similar probability density functions with finite mean and variance 
which maximize Renyi’s entropy. The investigation is restricted in the Schwartz space S(R d ) and 
in the space of Z-differentiable compactly supported functions C l c { R d ). Interestingly the solutions of 
this optimization problem do not coincide with the solutions of the usual porous medium equation 
with a Dirac point source, as it occurs in the optimization of Shannon’s entropy. We also study the 
concavity of the entropy power in R d with respect to time using two different methods. The first 
one takes advantage of the solutions determined earlier while the second one is based on a setting 
that could be used for Riemannian manifolds. 
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1 Introduction 

The last two decades have witnessed an enormous growing interest in using information concepts in 
diverse fields of science. Although Renyi entropy was introduced as early as 1961, only recently a 
wide range of applications has emerged as in the analysis of quantum entanglement 0, quantum 
correlations [4], computer vision [5], clustering [9], quantum cryptography [3] and pattern recognition 
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in the present work we solve three problems. The first examines the possibility to extremize 
Renyi’s entropy using self-similar probability density functions (p.d.f.’s) with zero expectation value 
and finite second moment in the Schwartz space and the space of compactly supported continuous 
functions on The second tackles the same problem but with the additional feature of non-zero 
mean. Finally, the third one is devoted to the determination of conditions under which the concavity 
of entropy power is valid. 

Our contribution is threefold: First, we theoretically establish the solutions of the first two problems 
by applying the method of calculus of variations, which was lacking from the literature. Second, we 
compare the specified solutions with the already known ones derived from the fast and porous medium 
equations. Third, we propose two different methods to answer the third problem. 

In particular, for the first problem, the functional which contains Renyi’s entropy and the con¬ 
straints, incorporated as Lagrange multipliers, is constructed and then by applying the calculus of 
variations its critical points is determined. The perturbed p.d.f.’s have the form g e = f + eh where 
|e| < eo < 1 and h are functions which are chosen in such a way that g e is a p.d.f. and has the same 
variance as /. The vanishing of the first variation of the Lagrange functional provides the equation 
which determines the critical points. It turns out that the solutions are unique. The nonnegativity of 
the second variation leads to an integral inequality which is preserved by the admissible perturbations 
we consider. Therefore the critical point is a local maximum of the functional. To prove its global 

1 On leave of absence from: Department of Mathematics, University of Aegean, School of Sciences, Karlovasi, 83200, 
Samos, Greece. 
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nature we use the concept of the relative Renyi entropy and examine its positivity at the critical point. 
This procedure can be generalised in R d and also by requiring a finite covariance constraint the well- 
known solutions of [21] are recovered. As a check one can prove that in the a —» 1 limit the solutions 
converge to the p.f.d. of the normal distribution jV(0, pf). The second problem is proved to be equiv¬ 
alent to the first one by performing a suitable transformation to the random variable (composition of 
a displacement with a rescaling). Therefore its solution maps to the one of the first problem. 

The knowledge of the solutions enables us to construct the nonlinear diffusion equation they satisfy 
and compare them with those of Zel’dovich, Kompaneets and Barenblatt (ZKB) [2], [22]. The difference 
is in the diffusion coefficient which depends not only on the shape and the size of a molecule but also 
on the order a of the Renyi entropy and the dimension of the space. We plot both our solution and 
the Barenblatt’s solutions and observe that their norms obey ||'K .|| 00 < ||/||oo! < a < «th. (d) 

while for values of a greater than the threshold a t h. (d) the inequality changes direction. 

Finally, the problem of concavity of the entropy power in W l is confronted by utilising two different 
methods. In the first method, the solutions of the first problem guarantee concavity on the condition 
that the second time derivative of Renyi’s entropy fulfils inequality (1691) . The second method is closer 
to the spirit analyzed in [23], where the a = 1 case was studied, and concavity holds provided that 
AMD is satisfied. 

The paper is organized into six sections. Section 2 reviews and proves some properties of the 
entropy. Section 3 determines the solutions of the two maximization problems using the method of 
calculus of variations and examines their global validity using the concept of relative Renyi entropy. 
Section 4 provides the nonlinear diffusion equation the solutions satisfy and compare it with the usual 
one. Section 5 proves the concavity of the Renyi entropy with respect to time following two different 
methods. Section 6 concludes the work and comments on more general constraints one could have 
considered. 

2 Preliminaries 

In this section we briefly review some properties of the Renyi entropy and for the sake of completeness 
we present the corresponding proofs. 

Definition 2.1 Let (Ll,A,p) be a probabilty space and an A-measurable function f : LI —$■ M + be 
a probability density function (p.d.f.). The differential Renyi entropy of order a, a £ M + , is the 
nonlinear functional 


H a :f^R 


defined by 


H a [f] : = 


l — a 


■ In 


dF 

dp 


a —1 


dF 


where F is the probability measure induced by f namely 

F(E) = [ f{x)dp{x), ME € A. 
J E 

Other equivalent ways of defining the Renyi entropy are 


H a [f\ 

H a [f} 


1 


1 — a 

1 


ln(E x (. f a ~\X))), Ma 

L a ) i 




(1) 

( 2 ) 

( 3 ) 


Properties 


l — a 


2 


Va > 1. 





(la) H a [f ] is continuous (a / 1) and strictly decreasing function in a unless f is the uniform density 
in which case it is constant. 

(1/3) A consequence of property (la) is the inequality 

0 < D KL (g\\f) < H^g] - (2 - a)H a [f], 

fa r 

where g= (g-f)dg> 0 

Jn f a dg Jn 

is the Kullback-Leibler (KB) relative entropy. If a 
reversed. Therefore the Renyi entropy as a function of a, for fixed /, is bounded by the difference 
between the Shannon entropy of g and the KB relative entropy of / and g. 

Proof 

(la) By Holder’s inequality there is a family of relations 

[ f^- 9)p+6q dg < ( [ f p dfi\ ( [ f q dp) (5) 

Jn \Jn J \Jn J 

holding whenever p, q > 0 and 9 € [0,1]. Taking p = 1 and assuming / to be a p.d.f. we 
have 

[ / 1 - 0+ ^< f [ f q dg) 6 (6) 

Jn \Jn J 

Let 0 < a < /3 < 1 and q = a. Then for 9 = (1 — /3)/(l — a) < 1 the previous inequality 

becomes 

[ fdp <([ f Q dp) 1_ “ (7) 

Jn \Jn J 

which, using that the ln-function is an increasing function, implies that 

Hp[f] < H a [f], 

The same proof holds for a, /3 > 1. 


for a < 1 


(4) 


and D KL (g\\f) = \ gln[j)dp 


> 1 then the direction of the inequality is 


(1/3) Differentiating H a with respect to a we obtain 

dHg[f\ = 1 ( TT m 1 

da (1 — a) \ a a f Q f a (x)dp 


Him 


= 77~ T (H a [f] + D KL (g\\f ) - -^l[/ a ]) 

a(l — a) \ a J 

= * , ((2 - a)H a [f] + D KL (g\\f ) - H^g]) < 0, Va < 1 

a(l — a) 

from which the inequality follows. 

(2) H a (f ) as a function of a converges to the following limits 

= ln(M^)) 

= #i[/] = - / /InM* 

Jn 

= ll/Hoo = inf.{0 < A < oo : / < A a.e. w G H}. 


lim #q,[/] 

i—>-0+ 

lim 3 Lq,(/) 

a —>^1 

lim H a [f] 


where H\[f] is the Shannon entropy of f. 


( 8 ) 


(9) 

( 10 ) 

( 11 ) 
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(3) Let / be a non-negative and integrable function w.r.t. the measure /i on f 1 , then the following 
inequality holds 

H a [f] < ln f dfJ ^ + ln (h(^)) for a < 1 ( 12 ) 

while for a > 1 the inequality is reversed. 

Proof 

This is a direct consequence of Jensen’s inequality 

<13) 

for concave functions <f>. In our case (cf) o f)(x)) = f(x) a is concave for a < 1 and by applying it 
we have 



from which it is deduced straightforwardly. In particular for a p.d.f. it reduces to H a [f] < 

\n(ii(n)). 

(4) If the Li(fi,R+) norm is invariant under the homogeneous dilations 

x —>■ x = A 7 x, A > 0, Mx € P C l d 


f(x) ->• fx(x) = A s f(x) 

(15) 

then H a , for a < 1, scales as 


H a [/a] = H a [/] - 5 ln A, 8 = dj. 

(16) 

Proof The L \ norm invariance of / 


11/ Li(n) = 11/ IlrA) = M 

(17) 

implies the condition 


5 = d'y 

(18) 


which combined with the definition of Renyi’s entropy produces the desired result. 


3 Formulation of the first problem and its solutions 

In what follows we restrict on the probability space (R, £>, dx) where B = <j(0) is the sigma algebra on 
open sets and dx the Lebesgue measure on R. The domain of the Renyi functional, T>(H a ), is defined 
to be 

V(H a< i) = j/€S(R) :jT/(s)ds = l| (19) 

V(H a>1 ) = |/ + € C c fc (R), k € Z+ and k < 

where / is a positive and integrable real valued function, f+(x) = /(x)xb(x) with \ the indicator 
function of the set B and [•] is the integer part of the number. 


f+(x)dx = 1 L 


( 20 ) 


4 












The first entropy maximization problem with vanishing mean and finite variance is formulated as: 


max.H a [f} subjected to the constraint E{X 2 ) = [ x 2 f{x)dx = \x\ 
f&v{H a< i) JR 

Using the method of Lagrange multipliers we construct the functional 

Hf;Ao,A 2 ) = #<*[/]- A fc (E(X fc ) -/4) , IMi = 1 

fce{ 0,2} 


= < oo. 


( 21 ) 


( 22 ) 


and impose appropriate conditions on the perturbations in order to calculate its first and second 
variation. 

Definition 3.1 A perturbation h G S(M) is called admissible if it satisfies the following conditions: 

\h(x)\ < c\f(x)\, 0 < c < 1, Mx G R, 
j h(x)dx = 0 and / x 2 h(x)dx = 0. (23) 

J M J M 

If we introduce the usual inner product in S'(R), the previous integral conditions imply that we search 
for a class of functions which are orthogonal to the unity and x 2 . Odd functions in the Schwartz space 
such as Q{x)e ~ b ^, b, fi > 0 with Q(x) be a polynomial of odd powers of x satisfies these criteria. 
Expanding the Lagrange functional in a Taylor series up to second order in e we obtain 


Hf + th; A 0 , A 2 ) = Hf\ Ao, A 2 ) + 5F f {h)e + 5 2 F f (h)e 2 + o(e 2 ). 
The first-order necessary condition for optimality requires }l2j 

F(f + eh; A 0 , A 2 ) — F(f\Xo, A 2 ) 


(24) 


SFt(h) = lim 


£—>•0+ 

= ^-F{f + eh- A 0 ,A 2 ) 


= 0, |e| < eo < 1 


(25) 


e=0 


or, equivalently, 


(f a 1 {x)~ Y' AfcX fc ^j h(x)dx = 0, where \ k = A fe g(a) = X k ——— [ f a (x)dx. (26) 

: V J a Jr 


k£{ 0,2} 

The function f(x) is determined by using the following lemma 
Lemma 3.2 If f G S( R) and if 


f{x)g(x)dx = 0, Mg G 5(R) such that lim g{x) = 0 

|ic|—>-oo 


(27) 


then f(x) =0, Mx G R. 

Proof 

Suppose that f(x) fi- 0 then there exists (gM such that /(£) = c > 0 (assuming that the constant is 
positive). Since / G S( R) there exists a neighbour (a, b) of £ in which f(x) > 0, Mx G (a, b). Define 
the function 


g{x) 


2 

{x - a)(b — x)e~ x X{a,b)( x )- 


(28) 
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The function g(x) is positive in (a, b) with g(a) = g(b ) = 0. However 


[ f{x)g{x)dx = I f(x)(x — a)(b — x)e x2 dx > 0 

JM. J a 


( 29 ) 


since the integrand is positive (except at a and b). This contradiction proves the lemma. 
Therefore f(x) is given by 


f(x) = 


J2 ~ x ^ k 

ke{ 0 , 2 } 


1 

1 — a. 


= A, 


V a 2 


1 

1 — Oi 


1+ ^, 


1 

1 — Oi 


(30) 


where the L°° norm is with respect to x and A*, > 0 since / E V a <\. 

In order / to be a local maximum the following second-order necessary condition for optimality should 
also hold 


d 2 


6 2 J r \ f (h)<0 or _^(/ + e h;A 0 ,A 2 ) 


< 0. 


(31) 


*=o ,/=/ 


In other words, the second variation of T at / should be positive semidefinite on the space of admissible 
perturbations h. The previous inequality is translated into 


a 


f a 1 (x)h(x)dx < ( / f a (x)dx 


a — 1 


f a ~ 2 (x)h\x)dx ) . 


(32) 


It is easily checked that the solution / together with the admissible perturbations satisfy the strict 
inequality, therefore / is a strict one-parameter family of local maxima with increasing Renyi entropy 
as property (la) of section 2 guarantees. 

Remark 

If 0 < f a (x)dx < 2 then Taylor expanding the ln-function around unity, the Renyi entropy reduces 
to the Havrda-Charvat entropy j7] also called Tsallis entropy 


S a (f) = 


a — 1 


1 — f a (x)dx 


(33) 


The solution in this case is the previous one but with the substitution A& = Afc ^ . 

Depending on the space of functions, we distinguish the following two types of solutions. 
(3a) The S(M ;) solution (a < 1). The requirement that / is a p.d.f. leads to 


1 1 

A 2 1 -“ 


0 Bi = \% for 0 < a < 1, 


while the second moment constraint gives 

Xq 1_a Bi = A|^ 2 (a) for i < a < 1. 


(34) 


(35) 


We adopt the abbreviation B s /2 = B(s/ 2,1/(1 —a) —s/2) from now on. From these two relations 
we conclude that 


Aq 


Aq / 1 — a 


1 


\ 2 Bi A 2 V3a-iy =fc f ° r 3 <0<L 


(36) 
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Using this relation, the exact solution can be expressed as 


/(*) = 


1 — a 


3a — 1J g 2 


1 + 


1 — a 
3a — 1 


x 


i 

1 —o: 


< a < 1. 


(37) 


This one-parameter family of local maxima of H a is unique and it remains to prove that it is 
actually also a one-parameter family of global maxima in T>{H a< \). For this we use the notion 
of the relative a-Renyi entropy of two densities / and g , defined by m 


Da(g\\f) = 


1 


l — a 


In / f a 1 (x)g{x)dx 


+ - —H a [g\, 

a a 


(38) 


where g satisfies the same second moment constraint as /. The first term in the right hand side 
of fgD equals to H a [f] as one may check since f R f a ~ 1 (x)g(x)dx = f R f a (x)dx. Therefore 

Da(g\\ f) = -(H a [/] -H a [g])> 0 (39) 

a 

by applying Holder’s inequality to the functions ( f a ~ 1 g) a and The same result holds 

in the a > 1 case. 


Proposition 3.3 The optimization problem for a < 1 has a one-parameter family of global 
maxima f which in the a —>• 1~ limit converges to the global maximum of the Shannon entropy 
which is the normal distribution AT (0, M 2 )- 


Proof 

The first two terms of (13711 in the a —>• 1~ limit give 


lim A 0 1 “ (a) 

Ot —^1 


1 


while the third term, by performing the change of variables s = (1 — a)/{3a — 1 ),p 
successively, converges to 




/ 3 ); 

lim ( 1 + s 7\ 
s ^o+ \ g 2 


lim 

p— >0+ 



1 + M P 


2s 


1 

P 


(40) 

l/2s 


(41) 


(3/3) The 1-differentiable compactly supported solution (ct > 1). 

similar to the previous one the solution turns out to be 


In this case following steps 


„_ 1 _ f (a — 1 A x^ \ a ~i 

f(x) = Aq -1 ^1 — - -j —j , a > 1 and x+ = max{ 0, x} = x9{x ), (42) 

where 6{x) is the unit-step function and the arbitrary constant Ao is specified as usual by 
imposing the requirement that / be a p.d.f. 


A~o(a) 


a — 1 


a—1 


3a — 1) ^ 5 ( 2 , — + !) 


(43) 
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Remarks 


• The solution (3a) can also be derived by integrating equation (1261) . in which case we obtain the 

relation between the Lagrange multipliers 

A 0 = --h'2'^2- (44) 

1 — a 

Eliminating Ao from the solution and applying the constraints we arrive at (1371) . 

• The derivation of (3a) and (3/3) is based on (lA.lft and (1A.2D which are connected through the 

transformation u = x 2 /(x 2 + 1) with u € (0,1), see (1A.3D . 

• The previous set up can also be applied to the more general case in which a covariance matrix 

C-constraint is present. The new solution can be derived from the old one by replacing x 2 by 
x T C~ 1 x , see (2TJ . 

4 Formulation of the second problem and its solutions 

The second entropy maximization problem with non-vanishing mean and finite variance is formulated 
as follows: 

max.H a [fx] subjected to the constraints E(X) = J R xfx(x)dx = m < oo 
fx&v(H a< i) (45) 

Var(X) = E(X 2 ) — /a 2 = /r 2 < oo. 

This problem can equivalently be restated as 

max.H a [fy] subjected to the new constraints E (Y) = f R yfy(y)dy = 0 
f Y eV(H a< i) (46) 

Var(y) = E(y 2 ) = 1. 

The random variables X, Y are related through the transformation X = + /t 1 and their corre¬ 

sponding p.d.f.’s fxify satisfy 

fx (x) = —fy (-——\ ■ (47) 

V 2 V M2 J 

As a consequence the entropies are given by 

H a [f x \ = H a [fy} + In 02 . (48) 

The solutions of the second problem (145(1 are therefore given by the solutions of first problem (|21l) 
with the substitution {x — y±)/ —>• x. 

One may also try to solve directly the second problem starting from the Lagrange functional 

Hf\ Ao, Ai, A 2 ) = Half] - Ao [J f(x)dx -lj- 2A 1 (E(X) - W ) - A 2 (Var(X) - /r 2 ), A fc € M. (49) 

The first-order necessary optimization condition dictates the solution 

2 

r-\x) = j2~^x k , ( 50 ) 

k =0 

which can also be proved to be a global maximum. 

We distinguish the following two classes of solutions. 










(4a) The S(R.) solution (a < 1). The positivity of the solution f a 1 (x), Vx € R requires A 2 > 0 
and A 2 A 0 — (A 1) 2 > 0. The p.d.f. and the mean value constraints lead to the condition 


/A = 


Ai 

A 2 


while the variance constraint implies 

Ao 

A 2 


(2n — 3)!! , 2 

»l + »l 


(2n- 5)!! 
Finally, the solution is written as 


n = 


<E Z + . 


f(x) = 


(2tt, — 2)!! /(2n — 5)!! 1 


(2tt, — 3)!! I/ (2tt, — 3)!! // 2 


1 + 


1 — a 


(2 n — 3)!! (x — [i\ 


(2n — 5)!! V 


(51) 


(52) 


(53) 


which is identical to the solution derived from the equivalent problem. 


(4/3) The 1-differentiable compactly supported solution (a > 1). In this case the polynomial 
)T) fc \kx k should be positive between its real roots. This occurs provided that A 2 < 0 and 
(A 1) 2 — A 2 A 0 > 0. Using the indicator function X(x\,x 2 )^ with ®i )®2 the roots of the polynomial, 
we find the previous solution with a relative minus sign between the terms inside the parentheses 
while the power is now positive. 


5 Comparison with the FME and PME solutions 


The p.d.f., /, which maximizes the Shannon entropy under finiteness of the second moment turns out 
to be identical to the fundamental solution of the diffusion equation with a Dirac point source. It is 
worth noting that / is actually a global maximum of H\. This observation is accidental as one may 
justify from the study of the corresponding optimization problem for the Renyi entropy. In particular, 
the nonlinear, initial valued problem 


d t u(x,t) 

u(x, t)dx 
i d 

has the following self-similar solutions ED 


lim 

t-> 0 + 


FME solution: u(x, t) a< 1 
PME solution: u(x, t) a> \ 


A u a (x,t), (x, t) € x ( 0 , 00 ) 
So(x) 


t~ s 

t~ 8 


C a < 1 + 


(1 — a)5 
2 ad 



(a — 1)5 
2 ad 



where 


5 = dy, 7 = 


Ca<l 

C*a> 1 


(2 + d(a — 1 )) ’ 




27 ( 1 - 0 ) 


, , d 

2 k 2 


1 1 d 

2 \ k \2 


27 ( 0 - 1 ) 


V s (ra)i^-ii 

|1 — ck| 


and 


K = 


2 ad 


(54) 

(55) 

(56) 

(57) 

(58) 

(59) 

(60) 
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The d-dimensional time-dependent functions 


1 ( x ^ \ a 

f(x,t) a< 1 = — A a<1 ( 1 + where /3 = /3(a,d) = 


^2 


11 — ck| 


(2 a — d( 1 — a)) 


and j 4 q ,<i = /?2 ■ 


if 


d 


d + 2 


< a < 1 , 


and 


/ ^*2 \ a _i 

f{x,t) a> i = —4*>i(i-/3- 

^2 V 


'awl 


where A a> i=(32- 




if ct > 1 


(61) 

(62) 


(63) 

(64) 


derived from the optimization of Renyi’s entropy, can be shown to satisfy the following initial value 
problem 


K a d t f(x,t ) = Af a (x,t), (x,t) € R d x ( 0 ,oo) 

lim ( [ f(x,t)dx J = 1 (65) 

t-> 0 + \jRd J 


provided that 7 = 7 (a,d) = 1/(2 + d(a — 1 )) and the coefficient is given by 


■ i 


2a(2 + d(q-!)) _! 

( 2 a + d(a — 1 )) 


( 66 ) 


where ^2 it) = t 7 ■ The presence of the ratio x/t 7 is implied by the self-similar property of the solution 
which requires the function into the parentheses to remain invariant under the rescalings: x —>• x = A 7 x 
and t —>• t = At. Therefore in general, the p.d.f. maximizing the Renyi entropy is a solution of an 
appropriately constructed difussion equation problem. 

We plot the FME and PME solutions of (|54l) versus the solutions of (1651) . 



Figure 1: A snapshot at t = 1 of the FME solution of (1541) (red line) and (|65l) (blue line) initial value 
problems, with parameters: d = 1, a = 3/4. 

In the fast diffusion case ||n||oo < H/lloo since C a< \ < A a< 1 , < a < 1 as one may prove using 

ra- 


io 














Figure 2: A snapshot at t = 1 of the PME solution of (15411 (red line) and (1651) (blue line) initial value 
problems, with parameters: d = 1, a = 2.2. 

In the porous medium regime ||tt||oo < ||/||oo up to the threshold value a t h .(d = 1) = 1.8268 for 
which C a> i = A a> i and then \\ul\oo > ||/||ooj Vcc > 1.8268. The threshold value a t h.> which depends 
on the dimension d, is determined arithmetically. 


6 The concavity of Renyi’s entropy power 


Definition 6.1 Let / : 0 x (0, oo) C x (0, oo) —> x (0,oo) be a p.d.f. in T> a . The a-weighted 

Fisher information of f is defined as 

wm = T -^— f ( 67 ) 

Jn f (] F Jn f 

while the entropy power of f associated to the Renyi entropy H a is defined as 


■M* [/](*) 


e (d + (“ 1 )) H a[f]( t ) = f a dflj 1 (J f a dfl) d ^~ a ) , 

< e §Hi [/] (t) ^ 

„ R{{x : f(x,t) > 0})3 -1 , 


0 < a < oo, a/1 
a = 1 
a = 0. 


( 68 ) 


Proposition 6.2 Let P = be the d-dimensional Euclidean space. The entropy power A f a is a 
concave function of t, Vf € (0, oo) provided that : 


d 2 H a [f](t) 
dt 2 


d +-2 
(2+d(ct—1)) L 

_ d _ J.—2 

(2a-l) 2 (2+d(a-l)) L 


*/ 3T2 < « < 1 
if a > 1, 


(69) 


where the right hand side of the inequality represents the contributions from the global maximum of 

H a . 


Proof 

Using the nonlinear diffusion equation, a straightforward calculation reveals the connection between 
dH a /dt and I a expressed by the relation 

ULMM = K-n a \f](t). ( 70 ) 
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The entropy power is a concave function of time iff 


d 2 N a [f](t) 


< 0 


+ (a — 1) 


dt 2 


dt 2 ” ' " V d 

or, equivalently when 

d 2 H a [f](t) —i dl(% [/] (t )) 


d?H a [f](t) (2 

+ j + (“ - !) 


d 


dH, 




dt 2 


= k: 


dt 




dH, 


dt J 


M*) V 


dt J 


Ma[f}(t) <0(71) 


(72) 


a 2 I \~ 1 

^ a 


Next, we establish the identity 

dH a [f](t) __ 

dt (1 - 2a) f Rd f a dx 

To do so we rewrite the integral as 

f I V/“ 12 


/ / 2a_1 A (In /) dx. 

J R d 


f 


-dx = 


/«-i V / a ■ V(ln/ a )cfe 
[ /«(V/“- 1 -V(lii/) + /“- 1 A(ln/)) 

/ V/ 2a_1 ■ V(In f)dx — a [ /^AQn /)di 

(2a; — 1) 

[ / 2a_1 A(ln f)dx. 

(1 - 2a) 


= —a 

a(l — a) 


The first time derivative of the Renyi entropy satisfies the following upper bounds 

: t~ 1 if ^ < a < 1 


dffq [/](*) 
dt 


< 


(2+d(a-l)) 

d 


(2a-l)(2+d(a-l)) 
To prove this note that the term A (In/) is given by 


A(ln f(x,t)) = —jz -—x 


-l 


d+2 

if a > 1. 


11 — a I t 2 '!' 


and therefore its contribution to the integral is 


r (i+zs-AT 2 

d + (d — 2 )^p7 


d-(d + 2)/3^ 


if 


d +2 


< a < 1 


if a > 1 


[ / 2a_1 A(ln f )d. 

JR d 


X = — ■ 


1 2/3 1 ~2|S d _ 1 | 

1 — a| t 2 7(i+ci(«-i)) 


x 


A 


2a—1 

a<l 


dB(i - I ) + (d ~ 2)B(1 + I, ^ - §) 


cZ a 


(73) 


(74) 


(75) 


(76) 


if 


< a < 1 


4 2a- 1 
yl a>l 


^(ra)-(<* + 2 ) s ( 1 + ra + 1 ) 


d a 


d +2 

if a > 1 


(77) 


Also, 


a 


A', 


-l 


«_ (2a + d(a-l)) x < 


(1 - 2 a) f Rd f a (x,t)dx 2(1 - 2 a) (2 + d(a-l)) |§d_i 


' rL a< 1 


S (f.^T+l) 


if 3T2 < « < 1 


if a > 1. 


(78) 
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Substituting (1751) into (1771) we recover the expected result. □ 

Note that in the a —> 1~ limit we reproduce the well-known result valid for the Shannon’s entropy. 

Next we prove the concavity of Reny’s power entropy on a different setting. This problem was also 
studied in [18] but our approach leads to a condition not predicted before. We will need the following 
lemma. 


Lemma 6.3 Let G be defined as 

r | V/ a|2 


Gaim) = 


f 


dx = 


f /|V 

JR d 


v\ 2 dx where v = 


a 


ca—1 


ol — Y 


then 


dG a [f](t) 

dt 

dlg[f]{t ) 
dt 


= - 2 K u ^ 

= (1 -a)K~ 1 I 2 a + 


1 [ /“ (||VVu || 2 + (a - l)(Au) 2 ) dx 

J R d 

1 dG a [f](t) 


f R d f a dx dt 

Proof 

Using the porous medium equations for f,v as well as the relation V/“ = fVv we have 


(79) 

(80) 
(81) 


dG[f]{t) 

dt 


(/t|Vu | 2 + 2ftVv(x,t) ■ Vv t ) dx, v t = 


dv 

dt 


K- 1 [ (A/ Q |Vu| 2 + 2 fVv -V((a- l)vVv + |Vu| 2 )) dx 

JR d 

K- 1 [ (A/“|Vu| 2 + 2 (a - 1) (/ \Vv\ 2 Av + fvVv ■ VAv) + 2 fVv ■ V|Vu| 2 ) dx 

J R d 

K~ x [ (Af a \Vv\ 2 + 2 (a - 1 )V/“ • VvAv + 2 af a X7v ■ VAv + 2V/“ • V|Vu| 2 ) dx 

jR d 

K~ l [ ((A/“ - 2A/“) |Vu| 2 - 2 (a - 1)/“ ((An) 2 + Vu • VAv) + 2 af a Vv • VAr) dx 


J R d 

K~ x [ (/“ (2Vu • VAr - A(|Vu| 2 )) - 2(a - l)f a (Av) 2 ) dx 


= -2K, 


: l [ r (||VVu || 2 + (a - l)(Au) 2 ) dx, 

JR d 


(82) 


where partial integrations in the fourth and fifth equalities have been used. Also in the last step the 
Bochner’s formula in Euclidean space has been applied. Relation (IHTT) is proved using (l80l) . □ 


Theorem 6.4 The Renyi entropy power, for self-similar solutions, is concave in t provided that the 
following inequality is satisfied 

1 


[ tno-c , / 2 “- 1 (OK> o, £ = c = 

JR d t' 


(83) 


a 2 (l + d(a — 1 )) ’ 

where the equality holds for a = 1 . 

Proof 

The Renyi entropy power is concave in t iff the condition ([72J holds which can be written equivalently 


as 


-i 


Illf](t) <d( rdx) / r (||VVu || 2 + (a-l)(Au) 2 ) 0 !x 

\JR d J J R d 

< d(l + d(a - 1)) ( f f a dx] [ f a \\VVv\\ 2 dx 

\JR d J JR d 

< d(l + d(a-l))J a [f](t), 


(84) 


13 









where ||VVu || 2 = ||Hessu || 2 = ( < % u ) 2 and we used the identity ||Hessu || 2 = Tr ((Hess 0 ) 2 ) > 

(Tr(Hess v)) 2 /d. Expanding || • || 2 and writing it in terms of In / the last integral can be casted into 
the form 


/ f a \\V\7v\\ 2 dx = a 2 [ f 2a 1 ^2 ((a-l)(dilnf)(djlnf) + dijlnf) 2 dx, (85) 

J R d J R d 


which by applying the Cauchy-Schwarz inequality we get that 

*_ 2 / ' N - 1 
d 

Using the identity 


/ riiwuii 

JR d 


o (X 

2 dx > 


f 2a ~ l dx^j (f Rd f 2al (( a — 1) IV In /| 2 + A In /) dx\ . 


AT 

f a 


= |V In f a \ 2 + A In / c 


we eliminate the |Vln /| 2 term from the previous relation and get 


(Jiu f a ) 


1 


Mm) >- 2 ,( 1 +^- 1 )) ^ 

In 1979, Aronson and Benilan obtained a second-order differential inequality of the form [U 
d 




2—1 


dxi 


-2 9f 

dx, 


a r- 1 1 > - 


a — 1 


(d(a 1 ) + 2 ) /.' a>a ° 1 1 d’ 


( 86 ) 

(87) 

( 88 ) 

□ 

(89) 


which applies to all positive smooth solutions of the porous medium equation defined on the whole 
Euclidean spac^j. In 1986 Li-Yau studied a heat type flow m on complete Riemannian manifolds 
(M, g ) with nonnegative Ricci scalar and for any positive function / on M and any t > 0 they arrived 
at the following sharp lower bound 

A(ln/)>-|, (90) 

pointwise, where A is the Laplace-Beltrami operator on M. An extension of the Aronson and Benilan 
estimate to the PME flow for all a > 1 and the FME flow for a € (a c , 1) on complete Riemannian 
manifolds with Ricci scalar bounded from below was given in Em¬ 
in our case these bounds change because our p.d.f. differs from the solution of the PME derived 
from the gradient flow of the functional m 

E[f](t) = —/ f a (x, t)dx. (91) 

I ~ a jR d 


7 Conclusions 

In this article we have proved that the p.d.f.’s that maximize Renyi’s entropy under the conditions of 
finite variance and of zero or non-zero mean are given by a one-parameter family of functions which 
belong to S(W l ) for a < 1 and to Cg(M rf ) for a > 1. This one-parameter family of functions is a 
global maximum of the entropy and satisfies the non-linear diffusion equation (|65jh The C oo norms of 
these solutions when compared to the corresponding ones derived from the fast and porous medium 

2 In d = 1 the restriction is a > 0 for general solutions. 
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diffusion equation initial value problem (1541) . they appear to behave in a particular way as seen in 
Figures (UJ) and ©. 

If one considers finite even moments of the random variable X, as constraints, and try to solve 
the corresponding maximization problem then the S( M) solution exists whenever f a ~ l is a complete 
polynomial of even degree, or equivalently, when all the coefficients \ 2 k+i vanish. In those cases 
there exists the possibility not to have interception points with the x-axis since the roots come into 
conjugate complex pairs. The compactly supported solution, under certain conditions, can always be 
determined. 

The concavity of entropy power holds whenever the second time derivative of the entropy varies 
according to (|69l) or the function f a — Cf 20-1 belongs to Li(M d ). It would be more appealing to 
have a deeper understanding of the origin of the later constraint which at this stage seems to be a 
requirement for consistency. 
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Appendix 


Using the two integral formulas [6] 

POO 

/ x^ii + x 2 y~ 1 dx 

Jo 

[ x^il-x^-'dx 
Jo 


l-i'-f), Re/i>0,Re(^ + |)<l 

Re/r, Rez^, A > 0, 

A A 


one can prove the following ones, used in section 3, 

1. 

poo -| pi 

/ x»- 1 {l + x 2 y- 1 dx = T U2-\1 -u) 

Jo 2 J o 

where we made the change of variable u = x 2 /(l + x 2 ) 




(A-l) 


(A.2) 


J§d—i 


A -1 


C 2 x ^,d , d N 


(n \ ^ \ 2 \\(a) drdS = l^- 1 !- where C ' 9 ' A >°, d < 2 H A -3) 

(C(a) + g(a)r 2 )X a ) 2 g 2 2 2 


d 

|Srf_i | = 2 j], is the volume of the unit ball and B(x,y) = r(x)r(y)/r(a: + y) is the Euler’s beta 
function. The constraint d < 2A for A = 1/(1 — a) allows only values 


a cn = 1-- < a < 1 for d > 3, 

0 d ~ ’ 


(A.4) 


while a co = 0 for d = 1, 2. 
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3. 



oo r d +1 

(C(a) + g(a)r 2 ) A (“) 


dr dS = 


,C 2 


+ 1-A 


i-ll 


2 g 1+ ~> 


--B(- + 1, A 


d 

2 


1) where d + 2 < 2A(A.5) 


The constraint between d, A becomes 


a, 


C2_1 d + 2 


< a < 1 for d > 1. 


(A.6) 


4. The last formula is 

[ n d -\C(a)-g(a)r 2 ) k + {a) drdS = |S d _i|^(a)-ic'(a)i +fc ^ J B(^ k(a) + 1) 

JSd-do 2 Z 

when k(a) > 0. (A.7) 

Formulas (lA.311 - flA.4ll are direct consequences of (1 A. 1 1) and (1A.2D . Another integral formula [61 used 
in section 4 is 


r x m 
k (ax 2 + 2/3x + c) n 


(—l) m TTa n ~ rn b m 
(2n — 2)!!(ac — b 2 ) n ~^ 

I" 1 / 2 ] /x / ,2 \ k 

x E (S) (2t ■ 1,!!(2n ~ 2k ~ 3)!! {^w~) 

where ac — b 2 > 0, 0 < m < 2(n — 1) 


and m, n are nonnegative, positive integers respectively. 
Identities involving the Euler’s beta function 


1 . 



Proof 

The large asymptotic expansion of the ratio of gamma functions m, is given by 


r (d + Q ) = o-6 

r( P + 6) p \ + 


b)(a + 6-1) 
P 


+ o(p" 2 ))- 


(A.8) 


(A.9) 


(A.10) 


Substituting the values a = — s/2, 6 = 0 in the previous expression and taking the limit p —> oo 
we recover the desired result. 


2 . 


d 


td( d 1 d\ 

l-OL 2 ) 

td( d a d\ 

^2’ 1—Q 2 ) 


d- 




+ (d — 2)' 
— (d + 2) 


gq+f a- j) 

73/ g£ a d\ 

^ 2 ’ 1 -a 2 ) 

B ( 1 + 4+T + 1 ) 

<* + !) 


-(2a -1) 
a 

d 

a 


(A.ll) 


These are direct consequences of the definition of the Euler’s beta function and the gamma’s 
function property T(x + 1) = xT(x). 
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3. 


D ,1 ^ (2n — 3)!! 

m ~2' n -2 ] = 

B(fc+-,n-fc--) = (2 k- 1)!! ^"~ ieZ* 

v 2’ 2 7 v 7 (2n — 2)!! 

fidn + 1) = 2 2n+1 i?(n + 1, n + 1) 

Proof 

Use the doubling formula for gamma functions and T(n) in terms of the double factorial 


(A-12) 


r [n + 


1 


(2?i — 1)!! 


r(n) = 


2 n 

(2n- 2)!! 


>71—1 


(A.13) 


Proposition 8.1 The function r(x)/T(x + c) with x, c € M + is decreasing in x. 
Proof It is enough to prove that 

f FbO \ 

= ip(x) — if(x + c) < 0. 


4-U r <*) 


dx \r(x + c) 
A straightforward computation gives 


- In 1 + 


x J x(x + c) 
by using the inequality 


< if(x) — '(/’( x + c) < — ln( 1 + 


x J 2x(x + c ) 


< 0 


lnx-< if(x) < lnx-. 

x 2x 


(A.14) 


(A.15) 


(A.16) 
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